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content-based appearance attribute comprises at least one of a luminance 
attribute, a chrominance attribute and a texture attribute.--. 



REMARKS 

In the Office Action, the Examiner noted that claims 1-21 are pending in the 
application and stand rejected. By this amendment, claims 12, 15 and 16 are 
cancelled, claims 1-5, 9, 13, 14, 17 and 21 are amended, claims 6-8, 10-11 and 
18-20 continue unamended, and new claims 22-26 are added. 

In view of both the amendments presented above and the following 
discussion, the applicant submits that none of claims now pending in the 
application is obvious under the provisions of 35 U.S.C. § 103. Thus the applicant 
believes that all of these claims are now in allowable form. 

Rejection Of Claims 1 . 2. 3. 1 1 . 1 2 & 21 Under 35 U.S.C. §1 03(A) 

Claims, 1, 2, 3, 11, 12 and 21 were rejected by the Examiner (per comment 
1 of the Office Action) under 35 U.S.C. §103(a) as being unpatentable over the 
Adelson patent (U.S. patent No. 5,706,417, issued January 6, 1998) in view of the 
Yeo, et al. patent (U.S. patent No. 5,821,945, issued October 13, 1998). The 
applicants respectfully traverse. However, the applicants have amended 
independent claims 1,19 and 21, along with various dependent claims, to more 
clearly claim the subject invention. 

Adelson notes that each object, set of objects, or portion of an object in the 
image having a motion vector significantly different from any other object in the 
image may be represented by a unique layer. Adelson teaches the representation 
of an image as a series of N layers ordered by "depth" in an image, where each 
layer comprises a series of data maps. Standard maps include an intensity map, 
an attenuation map, a velocity map, and a delta map. Optional maps include a 
contrast change map, a blur map, a depth map and a surface orientation map. 
Each map comprises a set of data for discrete two-dimensional locations and, 
optionally, a time dimension. There is no teaching of the use of a third dimension 
other than the depth associated with each of the end layers of an image. 

Yeo discloses a method for video browsing based on content and structure. 
The Yeo method arranges video information such that a human browsing through 
the arranged video information may easily find desired video imagery. Referring to 
FIG. 1 of the Yeo patent, scene change detection is employed to divide a video 
screen into a plurality of video "shots," which are then arranged into a plurality of 
"clusters," where each cluster comprises similar video shots. A hierarchical graph 
building technique is employed to provide a graphical means of transitioning 
between clusters or shots within clusters. In this manner, a browser may rapidly 
identify shots, or clusters of shots, having the desired video imagery (e.g., a 
particular speaker or a particular image). It is crucial to the understanding of the 
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present invention to note that the Yeo arrangement is not directed towards a 
layered representation of video or image information. Rather, the Yeo 
arrangement is directed towards the clustering of similar video imagery in a 
manner allowing rapid retrieval by a video browser utilizing a graphical metaphor 
to arrange and present the clustered video information. 

Adelson and Yeo, either singly or in combination, fail to disclose or suggest 
the invention per amended claim 1, which reads as follows (labels inserted to 
simplify the discussion): 

"A method for comprehensively representing video information in a 

manner facilitating indexing of the video information, comprising the step of: 

(a) segmenting a video stream into a plurality of scenes, each of said 
scenes comprising at least one video frame; 

(b) dividing, using intra-scene motion analysis, at least one of said 
plurality of scenes into at least one scene foreground layer and a scene 
background layer : 

(c) representing each scene background layer as a mosaic, said 
background layer mosaic defining a key frame of a respective scene : and 

(d) representing each of said at least one video frames forming said 
scene as a difference between initial video frame imagery and a respective 
portion of said key frame ." 

With respect to step (b), Adelson utilizes intra-frame motion to define layers 
within a video frame . By contrast, the subject invention claims "intra-scene motion 
analysis" to divide a "scene[] into at least one scene foreground layer and a scene 
background layer." Thus, Adelson provides a different structure, operating in a 
different manner to achieve a different purpose than the claimed invention. 

The teachings of Yeo do not bridge the considerable gap between Adelson 
and the claimed invention. Specifically, assuming arguendo that Adelson and Yeo 
were to be somehow operatively combined, the resulting combination would still 
lack the claimed element (b). That is, the resulting combination would, at most, 
provide for segmenting a video stream and processing individual frames within a 
scene to provide a series of frame-specific layers. 

With respect to step (c), there is absolutely no teaching of forming a mosaic 
within either of the two references, much less the claimed step of "representing 
each scene background layer as a mosaic." where the "background layer mosaic 
defin[es] a key frame of a respective scene." 

The Examiner contends that Adelson "teaches combining the foreground 
and background images to produce a video image, thereby implicitly teaching 
mosaic representation (column 2, lines 15-21; column 6, lines 50-55)." The 
Applicant respectfully disagrees, for at least the following reasons. 
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The portions of Adelson cited by the Examiner (page 2 and 6) notes that 
multiple motion vectors exist where, for example, the edge of a moving foreground 
object is blurry. Such motion blur and/or focus blur occurs in the case of, for 
example, an object such as a baseball moving rapidly across a display or viewing 
window. Simply put, the portions of text cited by the Examiner address intra-frame 
motion of a foreground object and the effect of the motion of that object on the 
clarity of traversed background imagery . This is entirely unlike the claimed 
invention, in which intra-scene processing is utilized to provide image layering for 
subsequent use in a mosaic representation. 

To clarify elements (c) and (d) of claim 1, the Examiner is referred to FIG. 7 
and the associated text beginning on page 20 of the subject application. 
Specifically, the graphical representation depicted in FIG. 7 is of a boat sailing from 
right to left. In the right most background scene 740, a sun 744 and clouds 746 
are found. In the left most background scene 730, a remainder portion of the 
clouds 736 and a dock 739 are found. The invention utilizes mosaic technology to 
combine the background images to produce a unified background image 760 
including the dock 769, a cloud 766, and the sun 764. This background image is 
used as a first frame in a sequence of frames depicting the scene of the boat 
sailing. The remaining frames F1 through F m of the scene 750 incorporates 
primarily foreground imagery of the boat moving within the frame. 

It is important to note that there is absolutely no teaching in the Adelson 
reference of combining background imagery from different frames to form a key or 
anchor background image which is then associated with a plurality of foreground 
images such as depicted in FIG. 7. The portion of text cited by the Examiner in 
support of his contention only supports the notion that a foreground object in 
motion tends to distort or blur background imagery proximate the object in motion. 
There is absolutely no teaching or suggestion within the Adelson reference that in 
the mosaic technique is employed in the manner described and claimed in claim 
1 of the subject invention. 

The applicant submits that it cannot be reasonably argued that combining a 
foreground and background image implicitly teaches a mosaic representation. 
This is because the mosaic representation of claim 1 inherently comprises a 
combining of at lest portions of multiple images, not of multiple layers within a 
single image (as provided by Adelson). 

The Examiner contends that Yeo teaches the "key frame" limitation of claim 
1. The Applicant respectfully disagrees, for at least the following reasons. 

The "key frame" described in column 1 of the Yeo arrangement is simply a 
single image frame representative of a first frame in a segment of video frames 
where a sequence of equal length segments, including respective key frames or 
first frames, is used to represent a video stream. Thus, the "key frame" of Yeo 
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comprises whichever video frame happens to be first within a segment of video 
frames. This key frame may be representative of the type of imagery in the rest of 
the segment or it may not (since a segment is defined as a fixed number of 
frames). Even if the Yeo key frame is, in fact, the first frame of a visually related 
scene, there is absolutely no teaching of a key frame formed according the 
claimed invention (as discussed above). 

It is noted that the Yeo arrangement utilizes scene cut detection to segment 
a video screen into a plurality of shots or scenes. However, the subject invention 
is not simply the dividing of a video stream into a plurality of scenes. Rather, the 
subject invention of claim 1 comprises a plurality of steps including the step of 
segmenting a video stream into a plurality of stream and, additionally, processing 
that step using various processing steps not shown in either of the Yeo 
arrangement or the Adelson arrangement. 

Since the references, either singly or in combination, do not disclose or 
suggest the claimed invention it is respectfully submitted that the invention of claim 
1, at least as amended, is patentable over the cited references. Moreover, since 
independent claims 17 and 21 include limitations similar to those found in 
independent claiml, it is submitted that claims 17 and 21 are patentable for at 
least the reasons discussed above with respect to claim 1. Therefore, the 
applicant submits that claims 1,17 and 21, as they now stand, fully satisfy the 
requirements of 35 U.S.C. § 103 and are patentable thereunder. 

Furthermore, all of the remaining dependent claims, including newly added 
claims 22-26 depend, either directly or indirectly, from claims 1, 17 or 21 and recite 
additional features therefor. As such and for the exact same reasons set forth 
above, the applicants submit that none of these claims is obvious with respect to 
the teachings of the cited references. Therefore, the applicant submits that all 
these dependent claims also fully satisfy the requirements of 35 U.S.C. § 103 and 
are patentable thereunder. 

Rejection Of Claims 4-8 Under 35 U.S.C. §1 03(A) 

Claims 4-8 were rejected by the Examiner (per comments 2 and 3 of the 
Office Action) under 35 U.S.C. §1 03(a) as being unpatentable over the Adelson 
patent in view of the Yeo, et al. patent as applied to claims 1 or 3 and further in view 
of the Burt et al. patent (U.S. patent No. 5,649,032, issued July 15, 1997). The 
applicants respectfully traverse. However, the applicants have amended claims 4 
and 5 and a discussion of the application of the Burt patent to the claims is 
appropriate. It is also noted that the Burt patent and the subject invention are 
commonly assigned to Sarnoff Corporation or its predecessor in interest. 

Burt teaches an image alignment system in which a plurality of related 
images, such as a sequence of video images, are aligned to form a mosaic. In an 
active mosaic example (see FIG. 9), four video frames showing a person walking 

10 



SN 08/970,889 



by some background objects are aligned to produce a mosaic. The mosaic is 
formed using the dominant image information in the video frames, namely the 
large frame area comprising background objects. The smaller image area 
devoted to the person walking can be defined as a residual error with respect to 
the mosaic. That is, background and foreground information is deduced using 
errors between majority (e.g., the background objects) and minority (i.e., the 
person walking) image elements. 

The Burt method is not the subject invention, nor does bridge the gap 
between the above-cited references and the claimed invention. Specifically, the 
subject invention (per claims 1, 17 and 21) includes the step of: "dividing, using 
intra-scene motion analysis, at least one of said plurality of scenes at least one 
scene foreground layer and a scene background layer." After the background layer 
of the scene is determined, the determined background layer is represented by a 
mosaic (which defines a key frame of a scene), while the foreground layer(s) are 
represented as "difference[s] between initial video frame imagery and a respective 
portion[s] of said key frame." Thus, background layers are subjected to mosaic 
processing, rather than being derived from such processing. 

As such, it is respectfully submitted that the Burt patent, alone or in 
combination with one or both of the cited references, fails to disclose or suggest 
the claimed invention. Therefore, the applicant submits that all the claims, as they 
now stand, fully satisfy the requirements of 35 U.S.C. § 103 and are patentable 
thereunder. 

Newly Added Claims 21-26 

Applicant has added new claims 22-26. It is submitted that the newly added 
claims do not introduce new matter and are fully supported by the specification. 
Applicant points out that newly-added claims 22-26 depend, either directly or 
indirectly, from claim 21 and include additional limitations thereto. Therefore, it is 
submitted that claims 22-31 are patentable for at least the reasons discussed 
above with respect to claim 21 . 

Formal Drawing s 

The applicant requests that the informal drawings initially submitted with the 
applicant be used during prosecution. Applicant will submit formal drawings upon 
receipt of Notice of Allowance. 
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Conclusion 



Thus, the applicant submits that none of the claims presently in the 
application are obvious under the provisions of 35 U.S.C. § 103. Consequently, the 
applicant believes all these claims are presently in condition for allowance. 
Accordingly, both reconsideration of this application and its swift passage to issue 
is earnestly solicited. 

If, however, the Examiner believes that there are any unresolved issues 
requiring the issuance of an adverse final action in any of the claims now pending 
in the application, it is requested that the Examiner telephone Mr. Eamon J. Wall 
Esq. at (732) 530-9404 so that appropriate arrangements can be made for 
resolving such issues as expeditiously as possible. 



Thomason, Moser and Patterson 

Attorneys at Law 

The Galleria 

2-40 Bridge Avenue 

P.O. Box 8160 

Red Bank, New Jersey 07701 

Please continue sending all correspondence to: 

Sarnoff Corporation 
201 Washington Road 
CN-5300 

Princeton, NJ 08543-5300 

Attention: Dr. William J. Burke 

Director, Law and Patent Operations 





Eamon / Wall, A 
Reg. No. 34,682 
(732) 530-9404 



12 



